1. Field of the Invention
This invention pertains to the field of pattern recognizers or artificial neural networks capable of supervised or unsupervised learning. More particularly, it pertains to such networks having neurons with digital weighting by a resistor network.
2. Description of the Prior Art
The general theory and operation of artificial neural networks (ANN) together with their existing and potential uses, construction, and methods and arrangements for learning and teaching have been extensively described in publicly available literature and need not be repeated herein. However, U.S. Pat. No. 4,951,239, which issued 21 Aug. 1990, and U.S. Pat. No. 5,150,450, which issued 22 Sep. 1992, describe aspects of artificial neural networks useful as a background of the present invention. Accordingly these patents, of which the present inventor is a coinventor, are incorporated herein by reference.
For the purposes of the present invention it is only necessary to realize that an ANN has a plurality of "neurons" each having a plurality of "synapses" individually receiving input signals to the network. Each synapse weights its received input signal by a factor, which may be fabricated into the network, may be loaded later as part of a predetermined set, or may be "learned". The synapses generate weighted signals which are summed by a common portion of the neuron, often termed the "neuron body", to generate a sum signal which, typically, is output by the neuron body after modification by a sigmoid or other "activation function" which is not directly involved in the present invention.
A practically usable ANN may have on the order of a hundred neurons each with on the order of a hundred or more synapses. It is, therefore, apparent that even with a very large scale integrated (VLSI) circuit it is important that the elements of each body and, especially, each synapse be of simple, compact, and regular configuration.
Prior art artificial neural network neurons have been implemented with a variety of weighting arrangements which, while generally effective, have various deficiencies such as lack of resolution, particularly over time and with repeated changes. This deficiency is avoidable by the use of synapses with digital weighting. However, since resolution to one part in 256--the integer two to the eighth power--or more is typically necessary for practical use of an ANN, it is apparent that providing such weighting at each synapse, as by field effect transistors (FET) having different widths corresponding to powers of two or some other number, requires elements of different sizes with the more significant digits having elements on the order of several hundred times larger than corresponding elements for the least significant digit. The use of such large elements and of such varying sizes in the synapses would result in impractically large circuits.
Also with VLSI circuits, temperature and fabrication variations across a chip cause different characteristics at different portions of the chip and may result in different synapses of even the same neuron having weight differences exceeding the necessary resolution. Such temperature and fabrication variations are particularly significant with transistors which have relatively complex structure requiring several integrated circuit layers participating in transistor functioning. As a result the characteristics of transistors tend to vary significantly with fabrication and temperature differences in the same integrated circuit. Transistors, in any event, have linear conductance characteristics only over relatively limited ranges which may impose undesirable limits on the dynamic range and useful voltage ranges of ANN employing transistors for weighting.
Further and since the synapse weights of an ANN may, for different applications, be unmodifiable after initial fabrication in a VLSI or other circuit; be generated after fabrication but not be readily changed after generation; or be readily changed, as by switching, when in use and as required in learning, it is essential that circuits providing such weights be adapted to not only provide the necessary resolution, temperature and fabrication variation immunity, and configuration for VLSI or other implementation; but be adapted to constructions providing the requisite fixed or modifiable weights.
ANN circuits overcoming certain of the above-described prior art problems and in meeting certain of the above-described requirements are disclosed in U.S. Pat. application Ser. No. 08/069,943, filed 28 May 1993 and which issued 27 Sep. 1994 as U.S. Pat. No. 5,350,953, and titled "DIGITALLY WEIGHTED NEURON FOR ARTIFICIAL NEURAL NETWORK". This prior filed application is commonly owned with the present application, has the present inventor as a coinventor, and is incorporated herein by reference. This prior filed application, Ser. No. 08/069,943, is characterized by an artificial neural network neuron wherein digital weighting of input signals is effected at a common portion of the neuron rather than at each synapse thereof and by the use of differential signals. This neuron of application Ser. No. 08/069,943 provides binary sign and bit selection by switching of input and reference signals at each synapse. In application Ser. No. 08/069,943 however, the weights of each synapse are provided by transistors which, although effective, have the before stated undesirable characteristics.
An integrated circuit resistor structure and method for forming this structure, which is also effective in overcoming certain of the above-described prior art problems and in meeting certain of the above-described requirements; which allows selectable switching; and which avoids the limited linearity and temperature and location related variations of transistors, are claimed in U.S. patent application Ser. No. 07/894,391, filed 5 Jun. 1992, titled "PROCESS FOR FORMING SYNAPSES IN NEURAL NETWORKS AND RESISTOR THEREFOR", and having Chi-Yung Fu as inventor. This prior filed application, Ser. No. 07/894,391, is also commonly owned with the present application and is also incorporated herein by reference.